1. Field of the Invention
The present invention relates to applications of wireless communications. In particular, the present invention relates to applications that can be implemented on a wireless handheld device to monitor wellness of a user of such a device.
2. Discussion of the Related Art
Stress is a key indicator of wellness in humans and is widely regarded as a prime contributor to poor performance and a cause of errors in various human activities. In the online article, entitled “Stress” (the “Panzarino Article”), by P. Panzarino, stress is referred to as a physiological change from homeostasis. The stress response can be thought of as the body's self-defense mechanism that responds to a physiological drift from homeostasis. The typical stress response includes the release of powerful neurotransmitters1 from the medulla in the adrenal gland. In response to stress, the adrenal medulla secretes two neurotransmitters: epinephrine (also called adrenaline) and norepinephrine (nor-adrenaline). The physiological effects of the fight-or-flight response (e.g., a rapid heart rate or a drop in temperature), for example, is caused by the release of these neurotransmitters. See, e.g., the on-line article, “Stress and Disease: New Perspectives” (the “Wein Article”), by H. Wein, 2006. Hence, one can gauge the human stress level by tracking a deviations in traditional physiological markers such as body temperature, heart rate, respiration rate, galvanic skin resistance (GSR), hormone concentrations, to name a few. See, e.g., the article, “Physiological Indicators of Stress in Domestic Livestock,” by D. C. Lay Jr. and M. E. Wilson, published in the Proc. Int. Anim. Agri. Food Sci. Conf., 2001. 1 Neurotransmitters are the body's chemicals that carry messages to and from the nerves.
Many diseases or conditions, including anxiety disorders, depression, high blood pressure, cardiovascular disease, certain gastrointestinal diseases, some cancers, and even the process of aging itself, may result from abnormal stress responses. Stress is also believed to be responsible for both the frequency and the severity of migraine headaches, episodes of asthma, and fluctuations of blood sugar in diabetics. Furthermore, scientific evidence indicates that people experiencing psychological stress are more prone to develop colds and other infections than their less-stressed peers. Overwhelming psychological stress (e.g., the event of 9-11) can cause both temporary and long-lasting symptoms of a serious psychiatric illness known as posttraumatic stress disorder (PTSD). See, e.g., the Pan Article.
Unpredictable, uncontrollable, and constant stress may have far-reaching consequence on one's physical and mental health. Stress can begin in the womb and may recur throughout life. One pathological (i.e., abnormal) consequence of stress is a learned helplessness that may lead to clinical depression. In addition, many illnesses (e.g., chronic anxiety states, high blood pressure, heart disease, and addictive disorders) also seem to be influenced by chronic or overwhelming stress.
Monitoring and managing stress can help relieve the side effects of stress. As demonstrated in the Wein Article, while stress is not always bad, too much stress is not good. Stress can help one stay focused, energetic, and alert. In emergency situations, stress can save one's life by bolstering self-defense (e.g., spurring one to slam on the brakes to avoid an accident). Therefore, it is highly desirable to be able to monitor and manage stress—or in general, wellness—using a simple and consistent method. A handheld device, such as a cellular telephone, can be an ideal instrument for implementing such a method, as practically everybody carries one such device at all times.
The study of human stress has involved scientists from different disciplines. Psychologists define stress resulting from emotions (i.e., as positive or negative reactions to situations consisting of events, actors, and objects). See, e.g., “The Cognitive Structure of Emotions” (the “Ortony Book”), A. Ortony, G. L. Clore, and A. Collins, Cambridge University Press, Cambridge, England, 1988. Physiologists demonstrate that high stress level often accompanies large deviations from normal conditions in processes such as heart beat, breathing, sweating, skin temperature, and muscle tension. Ergonomic studies uncover an “Inverted-U” relationship between stress and performance of a task. See, the article, “Worker Participation and Autonomy: A Multilevel Approach to Democracy at the Workplace,” B. Gardell, International Journal of Health Services 4:527-558, 1982. Further, facial expressions and gestures have been used to model an affective state recognition system. (See, e.g., the articles (a) “Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources,” J. Beatty, Psychological Bulletin 91:276-292, 1982; (b) “Robot in society: friend or appliance?” C. Breazeal, Proc. Workshop on Emotion-Based Agent Architectures, 18-26, 1999; (c) “Emotion and Sociable Humanoid Robots,” C. Breazeal, Int. J. Human-Computer Studies, vol. 59, pp. 119-155, 2003, (d) “Automatic Generation of Multi-Modal Dialogue from Text Based on Discourse Structure Analysis,” H. Prendinger, P. Piwek, and M. Ishizuka, Proc. Int. Conf. on Semantic Computing, 2007, pp. 27-36, and (e) “Recognition of Facial Expressions and Measurement of Levels of Interest from Video,” M. Yeasin, B. Bullot, and R. Sharma, IEEE Transactions on Multimedia, vol. 8, no. 3, pp. 500-508, June 2006.)
Various other approaches have been developed for recognizing user stress. For instance, stress in a car driver have been detected by measurements of physiological activities (e.g., Electromyograph (EMG), Electrocardiograph (ECG), respiration, and skin conductivity). Examples of such approaches are disclosed, for example, in (a) “Smartcar: Detecting Driver Stress,” J. Healy and R. Picard, Proc. 15th International Conference on Pattern Recognition, 2000, (b) “A Decision-Theoretic Model for Stress Theoretic Recognition and User Assistance” (“Liao Article I”), W. Liao, et al., Proc. AAAI, pp. 539-34, 2005; (c) “Toward a Decision-Theoretic Framework for Affect Recognition and User Assistance” (“Liao Article II”), W. Liao, W. Zhong, Z. Zhu, Q. Ji, and W. Gray, International Journal of Human-Computer Studies, 64: 847-873, 2006; and (d) “A reasoning-based framework for car driver's stress prediction” (the Rigas Article”), G Rigas et. al., Proc. 16th Mediterranean Conference on Control and Automation, 2008.
In the article, “Online Stress Detection Using Psychophysiological Signals for Implicit Human Robot Cooperation,” P. Rani, J. Sims, R. Brackin, and N. Sarkar, Robotica 20:673-685, 2002, the authors describe applying wavelet decomposition and fuzzy logic techniques to sympathetic and parasympathetic activities of a human heart to determine a stress level. The article, “Monitoring Driver Drowsiness and Stress in a Driving Simulator,” M. Rimini-Doering, D. Manstetten, T. Altmueller, U. Ladstaetter, and M. Mahler, Proc. First International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design. 58-63, 2001, describes combining physiological signals and expressive gestures (e.g., eye closure, head movement) to monitor driver drowsiness and stress in a driver simulator).
The variation in body temperature can be measured using inexpensive non-invasive methods. Non-invasive skin temperature measurements overcome the physical discomfort and the difficulty associated with taking measurements from a human body. See, e.g., the article “Development of a Skin Temperature Measuring System for Non-Contact Stress Evaluation” (the “Kataoka Article”), H. Kataoka, H. Yoshida, A. Saijo, M. Yasuda, and M. Osumi, Proc. 20th Annual International Conference of the IEEE Engineering in Medicine and Biology Society 2:940-943, 1998. The skin is kept alive by a minimal blood supply under control of an “internal thermostat,” the hypothalamus. In general, “relaxation” increases certain chemicals that increase the blood flow to the skin's surface, causing thereby an increased skin temperature, a drop in core body temperature, and increased heat transfer to the environment. See, e.g., (a) “The Thermosensitivity of the Hypothalamus and Thermoregulation in mammals” J. Bligh, Biological Reviews, vol. 41, no. 3, pp. 317-365, January 2008, and (b) “Principles of Science for Nurses” (the “James Article”), J. James, C. Baker, and H. Swain, published online. On the other hand, colder hands or feet reflect “activation” or “tension” due to activation of the body's sympathetic nervous system to make the muscles tensed and to speed up the heart rate and activities in other vital organs. Changes of 2-4 degrees Fahrenheit can occur in minutes. See, e.g., the James Article referenced above.
It has been observed that stress, whether emotional or physical in nature, can cause sharp fluctuations in body temperature. See, e.g., the articles, (a) “Fundamentals of Nursing: Human Health and Function,” R. F. Craven and C. J. Hirnle, 2008, (b) “Thermoregulation,” as published in Wikipedia, and (c) “Temperature of a Healthy Human (Body Temperature),” G. Elert, published online.
Research is ongoing on the relationship between a stressful task and skin temperature. One early work is a personal monitor developed to protect workers from heat stress, reported in the article, “Personal Monitor to Protect Workers from Heat Stress” (the “O'Brien Article”), J. F. O'Brien, T. E. Bernard, and W. L. Kenney, Proc. Conf. Human Factors and Power Plants, 1988. The O'Brien Article discloses that the authors developed a small device to be worn by a worker to assess both the body temperature and the heart rate and to alert the worker when either one of the assessments indicates excessive physiological strain. In the Kataoka Article, a facial skin temperature measuring system is reported for non-contact stress evaluation. The article “Development of Flexible Self Adhesive Patch for Professional Heat Stress Monitoring Service,” D. G Park, S. C. Shin, S. W. Kang, and Y. T. Kim, Proc. 27th Annual Int. IEEE Engineering in Medicine and Biology Society Conf., September 2005, discloses a flexible self-adhesive patch that could be worn on the chest for professional heat stress monitoring. In Liao Articles I and II, the authors demonstrated a high correlation between fatigue and a temperature measured for stress. In the article, “Wireless Temporal Artery Bandage Thermometer,” I. G Finvers, J. W. Haslett, and G Jullien, Proc. Biomedical Circuits and Systems Conference, 2006, Finvers disclosed a bandage-based thermometer for tracking wellness; the bandaged-based thermometer is placed on a patient's temple region of the forehead to measure the core body temperature in a non-invasive manner, to track the patient wellness.
However, these approaches measure single or multiple modalities and are only concerned with tracking the absolute values of physiological quantities. For example, the prior art focuses on mapping stress to an absolute reading of one or more physiological markers (e.g., the absolute temperature, absolute heart rate).
In addition to stress modeling, many articles disclose modeling affective states recognition. In addition to the Liao Articles I and II, such articles include (a) “Lifelike pedagogical Agents and Affective Computing: An Exploratory Synthesis,” C. Elliott, J. Rickel, and V. W. Friesen, Proc. Artificial Intelligence Today. Lecture Notes in Computer Science (1600), Springer Verlag, 1999, (b) “To feel or Not to Feel: The Role of Affect in Human-Computer Interaction,” E. Hudlicka, International Journal of Human-Computer Studies 59:1-32, 2003, and (c) “Probabilistic Combination of Multiple Modalities to Detect Interest,” A. Kapoor, R. Picard, and Y. Ivanov, Proc. International Conference on Pattern Recognition, 2004.
Recently, probabilistic-reasoning approaches have been applied to model user affects. The probabilistic-reasoning techniques are concerned with graphical models, such as hidden Markov models (HMM), Bayesian network (BN) and Influence Diagram (ID). See, e.g., Liao Article I. The book “Affective Computing” (the “Picard Book”), R. Picard, Cambridge University Press, Cambridge, England, 1997 discloses using an HMM to model the transitions among three affective states: interest, joy and distress. HMMs, however, lack the capability of representing dependencies and semantics at different levels of abstraction for affect modeling. The article “Active Affective State Detection and User Assistance with dynamic Bayesian Networks,” X. Li and Q. Ji, IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, 2004, discloses using a dynamic BN to recognize user affect and provide user assistance. However, this user assistance function is triggered by some pre-determined thresholds since a BN does not have an explicit representation for decision making (user assistance).
Likewise, the article “Probabilistic Assessment of User's Emotions in Educational Games,” C. Conati, Journal of Applied Artificial Intelligence, 16:555-575, 2002, discloses using a dynamic decision network to monitor a user's emotions and engagement during the interaction with educational games. However, their work uses only bodily expressions related features and suffers from a lack of validation. The dynamic Bayesian network (DBN) framework cited by Liao Article I uses dynamic inference and sequential decision making techniques to unify stress recognition with user assistance, utilizes evidences from multiple modalities, and is validated in a real-time system with theories of psychology.
The article “Development stress monitoring system based on personal digital assistant (PDA),” M. H. Lee, G Yang, H. K. Lee, and S. Bang, Proc. Engineering in Medicine and Biology Society, September 2004, pp. 2364-2367, discloses a stress monitoring feature in a wireless handheld device (e.g., a PDA). Lee monitors more than one physiological quantity, using an electrode to measure skin temperature. Lee merely tracks the skin temperature and infers stress levels based on the variation in the absolute reading of the temperature. Similarly, the article “A PDA based Ambulatory Human Skin Resistance Measuring System,” Q. Fang et. al., Telehealth, 2005, discloses a PDA-based ambulatory human skin resistance measuring system.
Only Liao Articles I and II and the article by Rigas et. al. disclose state-of-the-art temporal Bayesian network (TBN) frameworks for monitoring fatigue and stress. These frameworks are all multiple-modality frameworks that impose both hardware and computational burdens not suitable for implementing on a wireless handheld device. Further, while Liao Articles I and II provide a framework for inferring fatigue and stress using only mental stressors, the article by Rigas et. al., discloses a stress detection technique exclusively for a driving test.
In summary, the prior art teaches a number of stress-tracking models, which consider only instantaneous measurements of a physiological entity (e.g., the instantaneous body temperature, or the instantaneous heart rate).